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(57) New findings such as the highlight theory which 
determine the impression of a face are utilized for the 
technique of modifying a photographed facial image and 
the makeup technique. A method according to the 
present invention includes determining a desirable face 
desireci by a person who is subjected to facial modifica- 



tion or makeup, and performing image prcx;essing like 
shape-merging on an original facial Image or an image 
of a face of the subject person based on the desirable 
face to thereby prepare images of selectable modified 
faces. 
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Description 
TECHNICAL FIELD 

[0001] The present Invention relates to a technique s 
of nxxJifying facial images and a makeup art of putting 
makeifl3 on a face using the mocfifying technique. 

BACKGROUND OF THE INVENTION 

10 

[0OQ2] The present inventor has been making stud- 
ies, for a long time, on factors wrhich determine the faces 
of persons, particularly facial impressions, and beauty 
and ugliness, from the viewpoints of plastic surgery and 
cosmetic surgery. The study has provided several new is 
findings. One of them Is the "highlight theory". Conven- 
tionally, it was a general thought that the components of 
a face, such as eyebrows, eyes, a nose and lips, would 
become significant factors to determine a facial inpres- 
sion. The research done by the present inventor how- 20 
ever shows that the prime factor to detemilne a facial 
impression Is the shading state of a face. i.e. the states 
of a bright area and a dark area, particularly the state of 
the brightest area or highlighted area. The shading 
stete relates to shading on a face which occurs under 2S 
illumination of the natural light, and reflects or 
expresses the three-dimensional shape of the face. The 
stat^ of a highlighted area in such shading particularly 
affects a facial impressioa 

[0003] The analysis based on this highlight theory 30 
classifies faces into three general types. As exemplarily 
shown in Figure 5, the shapes of highlighted areas are 
an inverted triangle type [(a) in Figure 5], an oval type 
[(b) in Rgure 5) and a clover-leaf type [(c) in Rgure q. A 
typical beautiful face in each facial type has a good 3$ 
shape of the associated highlighted area. With regard to 
an inverted triangle type, for e)ample, no or little distor- 
tion or deformation on its shape is a significant factor in 
giving a good impression on a face. 
[0004] Another finding is a theory on the shapes of 40 
faces people feel beautiful or preferable. Conventional 
theories on facial beauty include canon theory or a the- 
ory based on golden cut. and the average theory which 
has recently come up. The average theory means a the- 
ory that the face that appears most beautiful among 45 
those of a certain human race is an average face of the 
individual faces of that race. Faces based on those the- 
ories merely provide general "beautiful faces". In other 
words, those theories cannot give ideal post-makeup 
faces which are makeup aims applicable to makeup so 
methods. Unlike those theories, the theory found by the 
present inventor can be put to a practical use in makeup 
methods. 

[0005J According to what has been newly found by 
the present inventor, through the following procedures. $$ 
it is possible in most occasions to find the most beautiful 
or preferable face lor each specific person, or the most 
beautiful or preferable face that can be made by 



makeip available to that individual. The fundamental 
theory is such that if faces are produced by combining a 
face of given sfandards, specifically, a face generally 
considered beautiful or a face preferred by an individual 
and ttie actual face of that particular individual at the 
proper mixing ratios tiirough an image processing tech- 
nique, an "ideal face" which is considered noost beautiful 
or preferable by that individual can be found in most 
cases from the foregoing mixed faces. Specifically, one 
of model faces, which are, for example, faces of pre- 
fenred talents or actresses or actors, or "beautiful faces" 
originated from the average theory and ttie like, is 
selected as a desirable face. Based on ttiis desirable 
face, the actual face of the person is subjected to image 
processing like morphing so that the actual face is 
mixed vnth the desirable face to approach ttie latter. 
This process provides a plurality of "modified faces" with 
different mixing ratios or similarity levels with respect to 
the desirable face. In most cases, those "modified 
faces" contain an Ideal face" tiiat person considers 
most beautiful or preferable. The data obtained by ttie 
present inventor shows that persons in 48 cas^ out of 
50 could find "ideal faces" ttiey ttiought most beautiful or 
preferable. Jn ttie otiier two cases, "ideal faces" could be 
found by modifying ttie desirable faces. The mixing ratio 
in every case ranged from 10 to 50%. 
[0006] It is to be noted that one model face itself 
should not necessarily be ttie most preferable face for a 
particuter person. That is. ttie "ideal face" a particular 
person can find for herself or himself is a "modified face" 
which is resulted from the mixing of a model face witti 
her or his own face, not ttie model face itself The actual 
results of the experiments conducted by the present 
inventor show ttiat most subject persons did not prefer 
mode! faces using "beautiful faces" created by the aver- 
age ttieory or the like. 

[0007] The present invention has been devised to 
effectively utilize the above-described findings and aims 
at using ttiose findings to a technique of modifying, for 
example, photographed facial images and to a makeup 
technique. 

DISCLOSURE OF THE INVENTION 

[0008] According to ttie present invention, facial 
images picked up by a still camera, a video camera or 
ttie like are modified as follows. First, a desirable face is 
determined, and ttien an original facial image is sub- 
jected to image processing to become closer to ttiis 
desirable face so that a modified facial image is 
acquired. 

(0009] Model faces can be used in detennining a 
desirable face The faces of preferable talents or 
actresses or actors can be used as model faces. If one 
does not have any preferable talent or the like, standard 
good-looking faces acquired from a plurality of beautiful 
faces which have previously been prepared correspond- 
ing to human races or like accorcfing to the foregoing 
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average theory may be used. In the foregoing method, ft 
may also be arranged that an original fadal image is 
subjected to image processing to become closer to a 
desirable face, thereby fornting Images of a pturaTity of 
selectable modified faces with different sintilarity levete 5 
with respect to the desirable face, and then a selection 
is made from those selectable nxxirfied faces to acquire 
a nxxfified fadal imaga This further fadfitates acquiring 
a face ideal to the sut)ject person. 
[001 0] ft 6 preferable to use the foregoing highlight 10 
tiieory in modifying and making an original fadal image 
doser to a desirable faca That is, the key point is to 
make the states of highlighted areas of a fadal image to 
be mocfif ied doser to those of a desirable faca Th^ can 
provide a more beautiful and preferable face and thus is 
fadlftate approaching to a desirable faca 
[0011] This facial image modifying method is an 
application of the foregoir^ theory on the shapes of 
faces that people think beautiful or preferaUa Unlike 
the conventional photograph modification technk^ues 20 
which mainly depend on a manual worK this method 
can automate ttie modification work by electronic 
processing. What is more, modification resufts can give 
greater satisfaction. This method of tiie present inven- 
tion can be adapted to operational simulation in plastic zs 
surgery and cosmetic surgery as well as ordinary pho- 
tograph modification. 

[0012] In a makeup simulation method embodying 
the present invention and a makeup method whidi uses 
this simulation metiiod, a desirable face is determined 30 
first. In the case where a n^keup instructor instructs a 
subject person how to put makeup, for example, a desir- 
able face may be detennined by asking the subject per- 
son about preferences. Afternatively. nrxxlel faces may 
be used in determining a desirable face. The faces of 35 
talents, or actresses or actors the subject person like 
can be used as model faces. If the subject person has 
no preference for any talent or the like, averages of 
good-looking faces which have prenously been pre- . 
pared corre^nding to human races or the like may be 40 
used as vnodeA faces. In tiie former case, a preferred 
desirable face can be selected from model faces using 
the name or the like of the preferable talent or the like as 
an index. In the latter case, a preferred desirable face 
should be selected from model faces of standard beauty 4s 
which has previously been irput as data. In tx3th cases 
a selection is made while sequentially displaying the 
model faces on a monitor screen or based on the type 
of the face of the subject person. In tiie latter case, the 
model faces should not necessarily be shown to tiie so 
subject person. 

[0013] Once the desirable face is decided, tiien 
images of imagined post-makeup faces based on the 
desirable face or post-makeup faces which are tnrag- 
ined as preferable are prepared. This involves a process ss 
of mixing the face of the subject person wrth the desira- 
ble face to make the former face doser to the latter by 
performing image processing like shape-merging on the 



image of the face of the subject person. Then, an ideal 
post-makeup face the sut^ect person desires is deter- 
mined from the imagined post-makeup faces. Spedft- 
calty. a pluralfty of imagined po^-makeup faces with 
diffa'ent mixing ratios or dmOarity levete witti respect to 
the desirable face are acquired through the foregoing 
inrage processing, and a preferable selection is made 
from those faces as an ideal post-makeup face witiiin 
the range of tiie similarity levels which can be detained 
by makeup. Thus, the ideal post-maket|> face which ts 
expected to be finally obtained is given in advance. That 
is. tiie subject person can know tiie makeup-finished 
face in a short tima As apparent from the above, the 
significant feature of tiie makeup mettiod embodying 
the present invention li^ in ttiat a final, makeup-finished 
face can be presented in a short time by determining a 
desirat>le face on which a makeup procedure is based 
and acquiring an ideal post-makeupfacewitii respect to 
tills desirable face. In this feature is effectively the 
above<lescribed ttieory on the shapes of faces people 
tfiink beautiful or preferabia 

[0014] When an ideal post-nrakeup face is dedded 
tiirough the foregoing makeup simulation, a makeup 
procedure is derived from this ideal post-makeup face. 
Specffioally, a sequence of makeup procedures for 
achieving the ideal post-makeup face, sudi as selecting 
which portions of eyebrows to be shaved and where to 
draw eyebrows, selecting tiie lines and ranges for eye 
liners and eye shadows, selecting tiie colors of eye 
shadows, setting the drawing range for a lipstick and 
separately applying different foundations, is derived 
from a previously set makei^D program. Based on ttiis 
makeup procedure, makeip is applied to tiie subject 
person. This makes it possible to accurately realize the 
ideal post-makeup foce or tiie makeup state the subject 
person has previously found appropriate, on the face of 
tiie subject person. That is. makeup whidi tiie subject 
person desires can be done freely and in a short tima 
[0015] To realize the foregoing feature of the 
makeup method of the present invention which lies in 
tiie pr^5aration of an ideal post-makeup face through 
image processing based on a desirable face or a model 
face, it is important to mix tiie cun-ent face of a subject 
person with a model face through image processing to 
get the former face doser to the latter. That is. ft 
essential to acquire an ideal post-makeup foce by mak- 
ing the cun-ent face of tiie subject person doser to the 
reference model face by image processing. The feature 
also requires an effective scheme or standards for mak- 
ing the cun-ent face of tiie subject person doser to tiie 
ideal post-makeup face by actual makeup. This can be 
accomplished by tiie foregdng highlight ttieory 
[001 6] Based on tiie highlight ttieory which teaches 
highlighted areas considerably affect the impression of 
a face and a beautiful and preferable face can be made 
by erasing any unnecessary shade (dari< area) to make 
ttie shapes of ttie highlighted areas better, it becomes 
relatively easy to get the face of a sutject person doser 
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to a model iaoe through image processing and it is also 
easy to make the face of the subject person closer to a 
desirable face by implementing such a makeup as to 
adjust the states of the highlighted areas in association 
with those of the nxxJei face (desirable face). In other 
words, a more effective scheme of making the face of 
the sut^ecl person closer to the reference model face 
can be acquired based oh the highlight theory. 
10017] The image processing for preparing the 
. images of imagined post-makeup faces on the basis of 
the highlight theory Involves a process of making the 
shapes of the eyebrows, the eye lines, the lip line, and 
so forth closer to those of a desirable face within a range 
where modification by makeup Is possible, in addition to 
the basic process of getting the states of highlighted 
areas on a face closer to those of the desirable face 
within a range where modification tjy makeup Is possi- 
ble. A shape-merging scheme or the like can be also 
used in this case. - - 

10018] Based on the highlight theory, attention 
shouW be paid to a greatest highlighted area as an 
inrpressing element of a face in the makeup simulation 
rnethod and makeup method, too. Since makeup about 
highlighted areas is accomplished by separately apply- 
ing different foundations, the types of foundations and 
procedures for applying them separately are derived as 
essential factors from an ideal post-makeup face. 
[0019] : To accomplish the process, image process- 
ing is perfonned to divide the ideal post-makeup face 
into a plurality of areas originating from differences 
between bright and dark areas WkB a partem of contour 
lines. Then, color data is acquired for each segmented 
area. Color data in this case consists of hue and bright- 
ness, and chroma if necessary, and R, G and B which 
are used in a television system or C, M. Y and K which 
are basic printing colors in a computer system are nor- 
mally used. Based on the color data of the individual 
areas and a previously set foundation program, founda- 
tions to be applied on the individual areas are deter- 
mined. Finally, the thud detemiined foundations are 
applied on the face of the subject person in accordance 
with the segmented areas. 

[0020] One makeup simulation method according 
to the present invention and a makeup method using 
this simulation method therefore Include the following 
steps: (a) determining a desirable face and perfomiing 
image processing on an image of a "face of a subject 
person based on the desirable face, thereby fbmiing 
images of imagined post-makei^D faces; (b) displaying 
the imagined post-makeup faces, prepared in the step 

(a) , on a monitor screen or the like and selectively deter- 
mining an ideal post-makeip face desired by the sub- 
ject person from the displayed imagined post-mateup 
faces; (c) deriving a makeup procedure from an image 
of the ideal post-makeup face, determined in the step 

(b) , based on a previously set makeup program; and (d) 
applying makeup on the face of the subject person in 
accordance with the makeup procedure derived in the 



8tep(c). 

[0021] Another makeup simulation method entxxl- 
ying the present Invention and another makeup method 
leing this simulation method, particularly, a makeup 
6 simulation method for applying makeup based on 6ark 
and bright states on a face and a makeip method using 
this simulation method, include the following steps: (e) 
detennining a desirable face and performing image 
processing on an image of a face of a subject person 
10 based on the desirable face, thereby forming Images of 
imagined post-makeup faces; [f) displaying the imag- 
ined post-makeup faces, prepared in the step (e), on a 
monitor screen or the like and selectively determining 
an (deal post-makeup face desired by the subject per- 
16 son from the displayed imagined post-makeup faces; 
(g) perfaming image processing to segment an image 
of the ideal post-makeup face, delemiined in the step 
(f). to a plurality of areas in regard to bright areas and 
dark areas; (h) acquiring color data for the segmented 
20 areas obtained in the step (g); © determining founda- 
tions to be applied on individual areas based on the 
color data for those areas, acquired in the step (h). and 
a previously set foundation program; and 0) separately 
applying foundations, determined in the step (i), on the 
25 face of the subject person in association with the indi- 
vidual areas. 

[0022] The makeup method based on dark and 
bright states may be modified in such a way as to be 
able to print the image of the ideal post-makeup face, 
30 segmented into a plurality of areas in regairl to bright 
areas and dark areas in the step (g), in an actual size of 
the face of the subject person, to cut out individual 
areas from the printed image to prepare a pattern mask 
for foundation, and to execute separate application of 
35 foundations in the step (k) by using this pattern mask. 
This design can allow different foundations to be sepa- 
rately applied on the associated areas easily and accu- 
rately That is. separate application of different 
foundations associated with an ideal post-makeup face 
40 can be carried out easily and accurately 

[0023] . ^ The makeup method which is based on the 
highlight theory may be modified in such a manner as to 
be able to project the image of the ideal post-makeup 
face, segmented into a plurality of areas in regard to 
45 bright areas and dark areas in the step (g). in an actual 
size on the face of the subject person, and to execute 
separate application of foundations in the step © based 
on this projected image. This modification can also per- 
mit different foundations to be separately applied on the 
50 associated areas easily and accurately 

[0024] Another makeup , method according to the 
present invention which is based on dari^ and bright 
slates includes the following steps: (A) determining a 
desirable face and performing image processing on an 
55 image of a face of a subject person based on the desir- 
able face, thereby forming images of imagined post- 
makeip faces; (B) displaying the imagined post- 
makeip faces, prepared in the step (A), on a monitor 
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screen or the Oke and selectively determining from the 
displayed imagined post-makeup faces an ideal post- 
makeup face the sut3ject person desires; (C) performing 
image processing to segment an image of the kleal 
post-makeup face, determined in the step (B). to a plu- 
rality of areas in regard to bright areas and dark areas; 
(D) acquiring cotor data for the segmented areas 
obtained in the step (C); (E) determining foundations to 
be applied on individual areas based on the color data 
for those areas, acquired in the step (D), and a previ- 
ously set foundation program; (F) printing foundation, 
determined in the step (E) in association with the pluraK 
ity of areas segmented in regard to bright areas and 
dark areas in the image processing in the step (C). on a 
thin base material to prepare a tranter film; and (G) 
transferring foundations, printed on the transfer film pre- 
pared in the tiep (i=). on the face of the subject person 
to thereby apply foundations, associated with the indi- 
vidual areas, on the face of the subject person. 
[0Q25] A further makei^ method according to the 
present invention which is based on 6axk and bright 
states includes the following steps: (H) detennining a 
desirai)le face and performing image processing on an 
irmge of a face of a subj^ person based on the desir- 
able face, thereby forming images of imagined post- 
makeup faces; (I) displaying the imagined post-makeup 
faces, prepared in the step (H), on a monitor saeen or 
the like and selectively determining from the displayed 
imagined post-makeup faces an ideal post-makeup face 
the subject person desires; (K) printing an image of the 
ideal post-makeup face in the bright and dark states, 
determined in the step (I), on a thin t>ase material using 
foundations to prepare a foundation transfer film; and 
(L) transfen'ing foundations, printed on tiie foundation 
trarisfer film prepared in the step (IQ. on the face of the 
subject person to thereby apply the foundations on tiie 
face of the subject person. 

[0Q26] Those melhods can repeatedly realize 
makeup for an ideal post-makeup face by canrying out 
simple procedures, such as transferring foundations 
from a transfer film accurately corresponding to founda- 
tions for the ideal post-makeup face and property shad- 
ing off the transfenred foundations at the boundaries 
between the individual areas. Thus, desired makeig) 
can be carried out more easily and in a shorter time. 
[0027] A makeup support apparatus according to 
the present invention for use in the foregoing makeup 
simulation methods arxi makeip methods comprises 
image storage means for storing images of a plurality of 
model faces; imaging and input means for imaging and 
inputting a face of a subject person; a monitor for dis- 
playing a necessary image; and processing means 
capable of performing a process of causing an image of 
the face of the subject person to approach a desirable 
face already determined to thereby form images of 
ioBgined post-makeup faces, a process of displaying 
tiie. imagined post-makeup faces on a screen of the 
monitor, arxi a process of deriving a makeup procedure 



from a determined ideal post-makeup face. 
[0028] It is preferable that tiie processing means tn 
ttie makeup support apparatus may further perform a 
process of segmenting ttie determined ideal post- 
5 makeup face to a plurality of areas in regard to bright 
areas and darti areas, a process of acquiring color data 
for ttie segmented areas, and a process of determining 
foundations to be appGed on individual areas based on 
ttie acquired color data for tttose areas. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0029] 

IS Figure 1 m a diagram for explaining one example of 
image processing in a facial image nxxJifyIng 
method. 

Rgure 2 is a structural diagram of a makeup sup- 
port apparatus according to one canrying-out mode 
20 of the present invwition. 

Figure 3 is an explanatory diagram concerning a 
highlight ttieory. 

Rgure 4 is a diagram for explaining segmentation 
into highlighted areas. 
25 Rgure 5 is an explanatory diagram of the types of 
highlighted are^. 

BEST MODE FOR CARRYING OLTT THE INVENTION 

30 [0030] Hereinbelow, canrying-out modes of the 
present invention will described. To begin with, a can-y- 
ing-out mode of a fadat image modifying method 
according to the present invention will be described. 

_ , Normally, a sy^em which includes a data processing 

35 apparatus like a personal computer and a monitor con- 
nected to ttie apparatus is used in implementing ttie 
facial image modifying metiiod of tiie present invention. 
An image of a face to be modified or an original facial 
image is put into the data processing apparatus. Then, 

40 a desirable face is selected from previously prepared 
model faces. e.g.. multiple standard faces which have 
been prepared based on ttie average ttieory. and ttie 
original facial image is subjected to image processing 
based on ttiis desirable face. 

45 [0031] The image processing is executed by a pro- 
gram which employs an image processing scheme like 
morphing or twining. Specifically, ttie coordinate values 
of facial constituents, such as eyebrows, eyes, a nose 
and a moutti. of both an original facial image M and a 

so desirable face T. and ttie ooorcfinate values of bright and 
dark patterns of each face are acquired as shown in Rg- 
ure 1. for example, and ttie original facial image M is 
mixed witti and transformed toward the desirable face T 
based on those ooorcfinate values. 

55 [0032] More specifically, first, essential portior^. 
such as eyebrows, ^es, a nose, a moutti and ttie lines 
of highlighted areas, are plotted for each of the original 
facial image M and ttie desirable face T. For example, a 
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tip portion of the nose is expressed as "0", the eye lines 
as "1 -2" and "3-4", and the eyebrows as "5-6" and "7-8". 
Then, those points on the original fedal innage M are 
shifted toward the corresponding points on the desirable 
face T by a morphing program. The lines of the high- 
lighted areas or the like are shifted or their inclinations 
are changed by a twining program. Such Image 
processing by shifting points or the like includes a plu- 
rality of phases according to the degree of such shifting. 
That is, selectable modified faces (imagined post- 
makei^ faces in the case of a makeqD method which 

will be described later) corresponding to those phases 
1:. can be obtained. If point shifting is perfomed every one 
tenth of the distance between con-esponding points on 
both facial images, for example, there are ten phases 
and there are thus ten imagined post-makeup faces. 
According to the experiments conducted by the present 
inventor, most of the times, subject persons found their 
"ideal faces" (selectable modified faces or imagined 
post-makeup faces) on selectable modified faces at the 
second or third phase, i.e., selectable modified faces 
which had approached to their desirable faces by 
degrees of 20 to 30%. That is. while there is a desirable 
face as an image, a face too close to this desirable face 
: comes off the actual "ideal face". 

As the above process forms the images of a 
plurality of selectable modified faces with different simi- 
larity levels with respect to a desirable face, they are dis- 
played on the saeen of the monitor to permit the subject 
person to make the best selection, which is treated as a 
modifiedfadal image.. . 

[0034] Now, canying-out modes of a makeup simu- 

~ lation method and a makeup method according to the 

/■ .P*^^®®®"* invention will be described. The makeup 
method c* the present invention is canried out by using a 
makeup support apparatus. The makeup suf^rt appa- 
ratus of which structure is illustrated in Rgure 2 includes . 
a dafa processing apparatus 1, a storage device 2, a 
monitor 3 and an image ser^ng unit 4. The data 
procesang apparatus 1, which executes various kinds 
data processing required In the makeup method of 
T the present invention, is a personal computer or the like. 
The storage device 2 Is used to store data of model 
faces which are used in the makeup method of the 
present Invention. An internal memory device or an 
. external memory device is used as the storage device 

[00351 In an exemplified case where a makeup 
i i instructor instructs a subject person a makeup proce- 
dure using such a makeup support system, the makeup 
method proceeds as follows. First, the instructor asks 
the subject person specific questions to determine the 
. : impression of a face desired by the subject person or a 
desirable face. Model faces are used to make a desira- 
l^e face concrete. As model faces are used Image data 
" of faces of popular talents, actresses or actors or the 

- - like, or standard good-looWng faces prepared in associ- 

- - fit»on with human races or the like, which are stored in 



the storage device 2. A desirable face is selected from 
the model faces by using an average face for each race, 
each era. each personality, each work or the like or the 
name of a spedfk: talent, or is selected from various 
5 model faces sequentially displayed on the screen of the 
monitor 3. 

[0036] Once the desirable face is determined, the 
face of the subject person is picked up by the image 
sensing unit 4 and Its image is sent into the storage 
10 device 2. By performing image processing on the image 
of the face of the subject person using the data process- 
ing apparatus 1. the image of the face of the subject 
person and the image of the desirable face are mixed to 
make the former image closer to the latter. In the image 
15 processing, the original facial image of the face of the 
subject person acquired in the above process is sub- 
jected to the same scheme as enriployed in the above- 
descnTsed modifying method to make the highlighted 
areas, the shapes of the eyebrows, the eye lines, the lip 
20 line and the like on the face approach those of the desir- 
at>le face step by step. 

[0037] Rgure 3 exemplifies highlighted areas, (a) in 
Figure 3 shows the states of bright and dark areas on 
the original facial image of the face of the subject per- 
25 son. and (b) in Figure 3 shows the states of bright and 
dark areas made close to those of the desirable face in 
image processing. In the figure, the densely dotted por- 
: tions represent relatively daric portions or darit areas 
and a portion surrounded by the dark portions represent 
30 the brightest area (highlighted area). As apparent from 
this example in which the highlighted area belongs to an 
oval shape, the current face of the subject person is off 
the ideal oval shape and the face of the subject person 
can be made closer to the desirable face which has an 
35 Ideal oval highlighted area by mainly ntKxlifying the off 
portions. 

[00381 As this process proceeds, the images of a 
pluralfty of imagined post-makeup faces with different 
similarity levels with respect to a desirable face are 
40 formed as desaibed above, and those images are dis- 
played on the screen of the monitor 3 to permit the sub- 
ject person to make a selection. If the subject person 
agrees with any of the displayed images, the selected 
face is set as an ideal post-makeup face. 
^ [0039] Once the ideal post-makeup face is deter- 
mined, a sequence of makeup procedures about por- 
tions of eyebrows to be shaved and where to draw 
eyebrows, the Knes and ranges for eye liners and eye 
shadows, the drawing range for a lipstick and separate 
so application of different foundations, is derived from the 
ideal post-makeup face. This sequence is carried out by 
a previously set makeup progranx 
[0040] With regard to foundation, in particular, how 
to apply foundations is derived by performing the follow- 
55 ing processes under the control of a foundation program 
which is included in the makeup program. First image 
processing is performed to divide the ideal post- 
makeup-face into a plurality of bright and dark areas. 



6 



11 



EP1030267A1 



12 



e.g., two to four areas. Th^ process is exenrpOfied in 
Figure 4. In the exenrpTified diagram, the face is seg- 
mented to four areas with different brightnesses (black- 
fiOed portion. densely<iotted portion, thinly-dotted por- 
tion and white portion). Next, color data for the seg- 5 
merited areas are acquired. The color data in this case 
are based on G and B used in a television systent 
Based on the obtained color data of each area« a fourv 
dation to be applied on that area is determined. 
[0041] As the makeup procedure needed for mak- 10 
Ing up the face of the subject person into the ideal post- 
makeup face and cosmetics to be used in the makeup 
procedure are determined in the above manner, actual 
makeup is finally applied to the face of the subject per- 
son based on what has been determined. In utilizing the is 
thus derived makeup procedure and the like in the 
actual makeup, data about the makeup procedure, etc. 
should be displayed on the monitor saeen or printed 
out 

[0042] According to a second carrying-out mode of 20 
the present invention, a pattem mask is used to apply 
foundations on the face of a subject person. A patten 
mask is formed like a pattern sheet by cutting out indi- 
vidual areas from a printed image of the face, seg- 
mented into a plurality of areas in regard to bright areas 25 
and dark areas in the above-described image process- 
ing, in the real size of the face of the sibject pers(»i. 
Then, the pattern mask is placed on the face of the sub- 
ject person and foundations for the individual areas are 
applied over the pattern mask. 30 
[0043] A third carrying-out mode of the present 
invention employs a projection scheme to apply fourxla- 
tions on the face of a subject person. Specifically, the 
image of the face, segmented into a plurality of areas in 
regard to bright areas and dark areas in the above- 3s 
desaibed image processing, is projected on the face of ' 
the subject person in real size, and foundations for the 
associated areas are separately applied based on this 
projected image. 

[0044] A fourth carrying-out mode of the present 40 
invention uses a transfer scheme to apply foundations 
on the face of a subject person. Specifically, first, in 
association with a plurality of areas segmented in 
regard to bright areas and dark areas in the atx3ve<jis- 
cussed image processing, foundations for the respec- 45 
five areas are printed in an actual size on a thin base 
material to prepare a foundation transfer film. Alterna- 
tively, images of bright and dark states on the ideal post- 
makeup face determined in the above-descnlsed man- 
ner are printed on a thin base material using founda- so 
tions to prepare a foundation transfer film. Then, such a 
foundation transfer fflm is pressed against the face of 
the subject person to transfer the foundatior^ Rnally. 
the proper shade-off process or the like is performed on 
the transferred foundations at the boundaries between ss 
the individual areas. The foundation transfer film in this 
case does not necessarily correspond to the entire face, 
but a partial transfer fOm for the upper eyelids or a por- 



tion around the nose, for example, may be used. 
[0045] According to the fourth carryingKnit mode of 
the present invention. appGcation of foundations to the 
face of the subject person is accomplished t}y directly 
printing the foundations on the face by means of a 
three-dimensfonal printer or the like. Spectfically, an 
image with separately applied foundations, virhich is 
formed by the above-des^foed inrtage processing, is 
direcdy spray-printed on the face by a three-dmerv 
sional printer or the like which uses the necessary foun- 
dations as inks. 

daims 

1. A facial image modifying method comprising the 
steps of determining a desirable face, and causing 
an original facial image to approach an image of 
said desirat)le face through image processing to 
acquire a modified facial image. 

2. A makeup simulation method comprising the steps 
of: 

(a) determining a desirable face and perfomv 
ing image processing on an image of a face of 
a subject person based on said desirable face, 
thereby forming images of imagined post- 
makeup faces: 

(b) displaying said imagined post-makeup 
faces, prepared in said step (a), on a monitor 
screen or the like and selectively deternuning 
from said displayed imagined post-makeup 
faces an ideal post-makeip face said subject 
person desires; and 

(c) deriving a makeup procedure from an 
image of said ideal post-makeup face, deter- 
mined in said step (b), based on a previously 
set makeup program. 

3. A makeup simulation method for executing makeup 
simulation based on shading states of a face, com- 
prising the steps of: 

(e) determining a desirable face and perform- 
ing image processing on an image of a face of 
a subject person based on said desirable face, 
thereby forming images of imagined post- 
maketqa faces; 

(f) di^laying said imagined post-makeup 
faces, prepared in said step (e). on a monitor 
screen or the like and selectively determining 
from said displayed imagined post-makeup 
faces an kleal post-makeup face saki subject 
person desires; 

(g) performing image processing to segment 
an image of said ideal post-makeup face, 
determined in said step (f). to a plurality of 
areas in regard to bright areas and daric areas: 
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(h) acquiring color data for said segmented 
areas obtained in said step (g); and 

(i) determining foundations to be applied on 
individual areas based on said color data for 
those areas, acquired in said step (h). and a 5 
previously set foundation program. 

The makeup simulation method according to daim 
2 or 3. further comprising a step of preparing a plu- 
ralit/ of model laces and determining said desirable io 
face based on said model fiaces. 

A makeup method comprising the steps of: 

(a) determining a desirable face and perform- is 
ing image processing on an image of a face of 

a subject person based on sakJ desirable lace, 
thereby fomrdng images of Imagined post- 
makeup faces; 

(b) displaying saki imagined post-makeup 20 
- faces, prepared in said step (a), on a monitor 

saeen or the like and selectively detennining 
from said displayed imagined post-makeup 
faces an ideal post-makeup face said subject 
person desires; > >t - 2s 

(c) deriving a makeup procedure from an 
image of said ideal post-makeup face, deter- 
mined in said step (b), based on a previously 
set makeup program; and 

(d) applying makeup on saki face of said sub- 30 
ject person in accordance with saki makeup 
procedure derived in said step (c). 

A makeup method for applying makeup based on 
shading states of a face, corrprising the steps of: 3S 

(e) determining a desirable face and perform- 
ing image processing on an image of a face of 
a subject person based on said desirable face, 
thereby forming Images of imagined post- 40 
makeup faces; 

(0 displaying said imagined post-makeup 
faces, prepared in said step (e). on a monitor 
SCTeen or the like and selectively determining 
from said displayed imagined post-makeup 4s 
faces an ideal post-makeup face said subject 
person desires; 

(g) performing image processing to segment 
an image of said ideal post-makeup face, 
determined in saki step (f), to a plurality of so 
areas in regard to bright areas and dark areas; 

(h) acquiring color data for said segmented 
areasobtained in sakistep (g); 

(1) determining foundations to be applied on 
Indivkiual areas based on said color data for ss 
those areas, acquired in saki step (h). and a 
. previously set foundation program; and 
0) separately applying foundations, determined 



in saki step (1), on saki face of said subject per- 
son in association with saki individual areas. 

7. The makeup method according to cfaim 6. further 
comprising: 

(k) printing saki image of saki kieal post- 
makeup face, segmented into a plurality of 
areas in regard to bright areas and dark areas 
in saki step (g) in claim 3, in an actual size of 
saki face of saki subject person; 
(I) cutting individual areas from a printed image 
obtained in saki step M to prepare a pattern 
mask for foundation; and 
(m) executing separate application of founda- 
tions in saki step Q) in cfaim 6 usir^ saki pat- 
tern mask prepared in saki step (I). 

8. The makeup method according to claim 6. further 
comprising: 

(n) projecting saki image of saki kieal post- 
makeup face, segmented into a plurality of 
areas in regard to bright areas and dark areas 
in the image processing in saki step (g) in daim 
3, in an actual size on sakt face of said subject 
person; and 

(o) executing separate application of founda- 
tions in said step (j) in cfaim 3 based on saki 
image projected in said step (n). 

9. A makeup method for applying makeup based on 
bright and dark states of a face, comprising the 
steps of: 

(A) detenmining a desirable face and perform- 
ing image processing on an image of a face of 
a subject person based on said desirable face, 
thereby forming images of imagined post- 
makeup faces; 

(B) displaying saki imagined post-makeup 
faces, prepared in said step (A), on a monitor 
screen or the like and selectively determining 
from said displayed imagined post-makeup 
faces an kieal post-makeup face saki subject 
person desires; 

(C) performing image processing to segment 
an image of said ideal post-makeup face, 
determined In said step (B). to a plurality of 
areas in regard to bright areas and dark areas; 

(D) acquiring color , data for saki segmented 
areas obtained in said step (C); 

(E) determining foundations to be applied on 
incfivkJual areas based on saki color data for 
those areas, acquired in said step (D), and a 
previously set foundation program; 

(F) printing foundations, determined in saki 
step (E) in association witii saki plurality of 
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areas segmented in regard to bright areas and 
dark areas in said image processing in said 
step (C). on a thin base material to prepare a 
foundation transfer film; and 

(G) transferring foundations, printed on said s 
foundation trans fe r film prepared in said step 
(F), on said face of saki subject person to 
thereby apply fourKiations. associated with said 
individual areas, on said face of said sibject 
person. to 

10. A makeup method comprising the steps of: 

(H) determining a desirable face and perfornr>- 

ing Image processing on an image of a face of 75 
a subject person based on said desirable face, 
thereby forming images of imagined post- 
makeup faces; 

(I) displaying said imagined post-makeup 
faces, prepared in said step (H), on a monitor so 
saeen or the like and selectively determining 

an ideal post-makeup face desired by said sub- 
ject person from said displayed imagined post* 
makeup faces; 

(K) printing an image of said ideal post-makeup 2S 
face, deterntined in said step (Q, on a thin t>ase 
material using foundations to prepare a foun- 
dation transfer film; and 
(L) transfening foundations, printed on said 
foundation transfer film prepared in said step 3o 
(K). on said face of said subject person to 
tiiereby apply said foundations on said face of 
said subject person. 

11. The makeup method according to any one of claims 35 
5 to 10. further comprising a st^ of preparing a 
plurality of model faces and determining said desir- 
able face based on said model faces. 

12. A makeup support apparatus for use In the makeup 4o 
simulation method as recited in any one of daims 2 

to 4 or in the makeup method as recited in any one 
of clain^ 5 to 1 1 , comprising: 



a determined ideal post-makeup face 

13. The makeup support apparatus according to daim 
12, wherein said processing means further per- 
forms a process of segmenting said determined 
ideal po^-makeup face to a pluraTrty of areas in 
regard to bright areas and dark areas, a process of 
acquiring color data for said segmented areas, and 
a process of determining foundations to be appGed 
on individual areas based on saki acquired color 
data for those areas. 

14. A foundation transfer film for use in separate appli- 
cation of foundations, saki foundation transfer film 
teving foundations printed on a thin base material 
in regard to bright areas and dark areas on a faca 



image storage means for storing images of a 45 
plurality of model faces; 
image pick-up and input means for imaging and 
inputting a face of a subject person; 
a monitor for displaying a necessary image; 
and so 
processing means capable of perfcn'ming a 
process of causing an image of said face of 
said subject person to approach a desirable 
face already determined to thereby form 
images of inr^gined post-makeup faces, a 55 
process of displaying saki imagined post- 
makeup faces on a screen of said monitor, and 
a process of deriving a makeup procedure from 
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